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Designing a 12.81 bps Fixed 
Box Router 


As the bandwidth of silicon increases over time it has become possible to build high 
bandwidth routing platforms in smaller form factors. In 2014, Cisco announced the NCS 

6008 48RU, an 8Tbps core router which consumes 11kW of power during normal operation. 
With the release of Cisco Silicon One™ in December 2019, we announced the Cisco 8201, a 
10.8Tbps 1RU router which consumes a mere 415W, enabling an incredible 48 times reduction 
in rack space and a 26 times reduction in power from what we shipped just five years before, 
well out-pacing Moore’s Law. As we in December 2019, this system is five times 
more power efficient than its closest competitor. 


With the release of the Cisco Silicon One Q200, a 12.8Tbps piece of routing silicon, Cisco 
enables its silicon customers to build even higher performing and more power-efficient routers 
than we did just nine months earlier. 


In this paper we will study some of the ways Cisco Silicon One achieves such a huge lead over 
others in the industry and how, for the first time, routing silicon consumes less power than the 
optics which connect into the system. We will also examine how this may translate to up to a 
$150,000 savings over a 10-year lifespan for every 12.8Tbps system deployed at a colocation 
facility. This potential electricity cost savings is accompanied by less environmental impact 
associated with decreased material use, reduced logistics costs from the smaller footprint and 
substantially fewer Greenhouse Gas (GHG) emissions from operation. 


© 2020 Cisco and/or its affiliates. All rights reserved. 


Cisco public 


Table of Contents 


Introduction 

Building a fixed-box router 
Latency savings 

Power savings benefits 


Conclusion 


© 2020 Cisco and/or its affiliates. All rights reserved. 


Building a fixed-box router 


There are many pieces of routing silicon available on the market today from 
both third-party providers and systems companies. Currently the highest 
performance routing silicon besides Cisco Silicon One is a 7.2Tbps device. 
We believe that to get equivalent IPv4 Longest-Prefix-Match (LPM) scale up 
to two external lookup engines may be needed. 


Consequently, with Cisco Silicon One we can build a 12.8Tbps router with a 
single piece of routing silicon, while with other devices you can only build a 
lower scale 7.2Tbps (18x400G) system or a high scale 6.4Tbps (16x400G) 
system with external lookup engines. 
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Figure 1. Fixed box with single device 


To achieve higher bandwidth, others need to use two devices connected 

in a back-to-back configuration which can more than double the latency, 
cost, power, and rack units, magnifying the benefits of Cisco Silicon One and 
demonstrating how advanced our architecture is. 
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Figure 2. High bandwidth fixed box 


Latency savings 

A consequence of having back-to-back devices is that packets need to travel over multiple hops as they traverse 
from an input port to an output port. As packets arrive at the ingress device, they must be routed over Serializer/ 
Deserializer (SerDes) to the egress device, adding significant additional latency. 


With Cisco Silicon One, the packets are switched locally within a device without ever going to another device. This 
means that Cisco Silicon One can forward packets with less than one half the latency of other solutions. 


Other Other 
Silicon One Routing Routing 
Q200 Silicon Silicon 
12.8T | | 


7 


RX TX 


Figure 3. Latency savings 
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Power savings benefits 


The power penalties of other architectures come from several places: 


- Impacts due to the silicon itself 

- Increase in silicon core die area (two to six devices versus one device) 

- Additional SerDes to connect the chips together 

- Reading the packet from buffers and move the data multiple times 

« Secondary Impacts 

« Increased loss in power planes delivering the higher current to the chips 

- Increased loss in power supplies delivering the higher current to the system 
« Increased fan speed to cool the higher power draw 


These impacts translate to a very significant effect at the full system level. Systems built with the Cisco Silicon One 
Q200-based system can provide 12.8Tbps of routing performance in under 390W of power, while we estimate 
systems built with the next closest competitor consume more than 1,000W and with high-scale Longest Prefix Match 
(LPM) we estimate it requires more than 1,350W. 
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Table 1. System power 
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Figure 4. System power 
In fact, the power of Q200 is so low that for the first time in history the power consumed for all the routing silicon in 
the system is below the power consumed by the optics, even when using power efficient 400G optics. 


Looking at Figure 5 one can clearly see that Cisco Silicon One fundamentally changes the game providing all the 
complexity of ethernet, packet forwarding, buffering, and scheduling in under the power it takes to send light down a 
fiber. 
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Figure 5. Silicon vs. optics power 


System power consumption impacts customers in two key ways: 


- Facilities need to be provisioned to deliver the power to the systems 
- Facilities need to be built to cool the heat generated by the systems 


The efficiency with which a facility can deliver the power to the systems and remove the heat is known as Power 
Usage Effectiveness (PUE). The most advanced web scale data centers in the world are approaching a PUE of 1.1 
(which is equivalent to a 10 percent ‘tax’ to deliver the power and cool the heat generated), while some older service 
provider facilities have PUEs above 3.0. We believe that a value of 1.7 PUE is a reasonable estimate for a modern 
service provider facility in the United States. 


We created a model to run an analysis of how much it would cost to deploy these systems over a 10-year life at a 
colocation facility. These facilities have a few fee structures for deploying equipment which include the size of the 
equipment, the amount of power draw, and the power redundancy model. The rental of the rack space costs between 
$50 and $300! per month per Rack Unit (RU). To be conservative, for the analysis we’ll assume the cost to be on the 
low side with $100 per month per RU. The primary power delivery costs approximately $3502 per month per kW and 
redundant power delivery we estimate to be $175 per month per kW. 
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With these assumptions we found that customers could potentially save $143,000 over a 10-year span for deploying 
a single Cisco Silicon One fixed box system. 
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Figure 6. Estimated colocation cost for a single 12.8Tbps router 


Possible 10-year cost of , 
System deployment for one system at Potential added expense 
a colocation site 


PT PAY $156K $87K (2.27x) 
2x6.4T with lookup engines $212K $143K (3.09x) 


Table 2. Estimated 10-year savings 


Conclusion 


Cisco Silicon One ushers in a new era of networking silicon, enabling power efficiencies which were unheard of in 
the past. The radical reduction in silicon to build a 12.8Tbps system impacts the cost, power, latency, and space for a 
router which can help provide significant cost savings to the network operators who deploy the equipment. 


Learn more 


Visit Cisco Silicon One 


1. https://pbhoenixnap.com/blog/colocation-pricing-guide-to-c 


2. https://www.365datacenters.com/portfolio-items/data-center-colocation-build-vs-buy/ 
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